1. Field of the Invention
The present invention relates to an improved three-dimensional object surface shape modeling technique. More concretely, the invention relates to an improved three-dimensional object curved surface model data generating technique. Particularly, the invention relates to an improved curved surface model generating technique which makes use of both advantages of a modeling method using distance image data and a modeling method using silhouette data.
2. Description of the Related Art
<Method of Directly Obtaining a Distance Image by Means of Three-dimensional Shape Input Device>
A three-dimensional shape input device, which emits a laser or a pattern light onto an object and capturing a light reflected from a surface of the object by means of a sensor so as to thickly sample and input three-dimensional shape data of the tridimensional object (distance image data), has been used a lot in fields of CG and industrial design (there is a device which does not carry out projection, but its accuracy is low). One of its reasons is that a shape of a real object can be introduced into a virtual space by using a three-dimensional shape input device and thus a desired three-dimensional model can be generated simply.
However, since the above three-dimensional shape input device utilizes an irregular reflection property of a light on a surface of the object, the device cannot input shapes of an object such as black hair with low surface reflectance, an object such as metal having a surface causing mirror reflection and a (semi-) transparent object such as glass properly, or their data are omitted (also in a device which does not carry out projection, the similar phenomenon occurs).
<Method of Utilizing Silhouette Data>
In this case, a Shape from Silhouette method of estimating a shape of an object from a shielding outline of the object is used a lot. One of those methods which is used the most is a Volume Intersection method of obtaining an area of a 3D space within a silhouette of the entire image. This area is called as visual hull, and the most general method of obtaining a visual hull will be explained below.
As shown in FIG. 11, an object is photographed from plural angle positions, and positions and postures of the camera are recorded (in the drawing, angles 1, 2 and 3 in the plural angles are shown). As shown in FIG. 12, a lot of cubic elements which are called as voxel in a virtual three-dimensional space are arranged. Here, FIGS. 11 and 12 are tactically two-dimensional schematic diagrams.
The voxels are projected on all images by using the recorded positions and postures of the camera, and only voxels in all the voxels which are projected within an area in all the images corresponding to the object are connected so that the shape of the object is restructured.
In the case where the shape of the object is restructured by the Shape from Silhouette method, one of the points which influences accuracy is a number of angles to be input. Particularly to a free-form surface object since only part of a sculptural surface of an actual object contacts with respective silhouettes, a 3D model to be generated has a shape which swells slightly to the outside except for the contact portion. FIG. 13 shows a portion which is recognized as the swelled shape (dummy information). In order to solve this, the images of the object should be input from a lot of angles.
However, a user should do a lot of works for inputting the images of the object from a lot of angles. It is considered that the images of the object are input from a lot of angles automatically by using turntable, robot arm or the like, or a lot of cameras are previously provided around the object. However, with these methods, size and shape of an object are limited strictly, and an apparatus becomes expensive.
<Another Conventional Method>
Therefore, for example, Sullivan and the others suggest a method of fitting a curved surface model directly to a silhouette and generating a curved surface model which contacts inside with the silhouette. With their method, a smooth property of a spline curved surface is utilized so that a 3D shape of an object can be generated by using input images from less angles.
Here, it is important that a curved surface model is not fitted to 3D data generated by the Shape from Silhouette method and the curved surface model is fitted directly to the silhouette. As a delicate method of generating a smooth curved surface model from a silhouette, it is considered that a curved surface model is fitted to 3D data generated by the Shape from Silhouette method, but in the case where a number of input angles is small, great distortion occurs. As mentioned above, this is because the 3D data generated from silhouette images from particularly less angles by the Shape from Silhouette method include a lot of dummy information.
However, even if this method is used among the object surface, a shape of a recessed portion of an object which does not appear on any images as a silhouette can not be reproduced properly. Namely, in the case where shape data are input by a three-dimensional shape input device, as mentioned above, there arises a problem that omission of data occurs on a black hair portion and a metal surface. Meanwhile, in the case where 3D data are generated from silhouette images, there arises a problem that a recessed portion cannot be reproduced.
Therefore, Japanese Patent No. 2601607 suggests a method of inputting shape data by means of a three-dimensional shape input device and simultaneously independently estimating a shape from silhouette image so as to select one of the shapes as shape information according to reflectance on the object surfaces on the respective portions.
However, this method has a disadvantage that since a shape is independently estimated from a silhouette, a distortion such as a step easily occurs in a boundary with the portion input by the three-dimensional shape input device. Moreover, with this method, since a curved surface is not used for estimating shape data from a silhouette, when the shape is reproduced accurately, input from a lot of angles is necessary.
In order to prevent the generation of a step, it is considered that a curved surface is fitted to or blended with a shape measured by a three-dimensional shape input device and a shape which is independently estimated by the Shape from Silhouette method simultaneously. However, as mentioned above, since the 3D data, which are generated from the silhouette images from particularly less angles by the Shape from Silhouette method, include a lot of dummy information, this method does not produce good results.